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Abstract: Collaborative gestures in the mathematics classroom occur when multiple learners 
coordinate their bodies in concert to accomplish mathematical goals. Collaborative gestures 
show how cognition becomes distributed across a system of dynamic agents, allowing for 
members of groups of students to act and gesture as one. We explore ways high school 
students gesture collaboratively while proving geometric conjectures in the sociotechnological 
context of an embodied video game in their classrooms. We find that students use their bodies 
and joint action to reason together and establish intersubjectivity. Gestures can both allow 
learners to access mathematical ideas and exclude peers. We also find that gestures generally, 
and collaborative gestures specifically, are associated with powerful forms of mathematical 
reasoning about generalizable properties of space and shape. 


Gestures — learner-generated movements of the body that often accompany speech to convey ideas or emphasis 
— have been identified as important to the learning of mathematics (Alibali & Nathan, 2012). Theories of gesture 
production examine these behaviors as both individual cognitive processes (e.g., Hostetter & Alibali, 2008), and 
as a significant aspect of collaborative processes, as learners work together to solve mathematical and spatial 
problems (Enyedy, 2005; Hall, Ma, & Nemirovsky, 2015). Extending notions of depictive gestures that use 
iconicity to invoke mathematical entities and relations (Streeck, Goodwin, & LeBaron, 2011), Walkington and 
colleagues (in press) created a taxonomy for different ways gestures can be used collaboratively — including one 
learner echoing a gesture that another learner has previously made, one learner silently following along and 
mirroring another learner’s gesture as it happens in real time, and joint gestures where two learners place their 
hands together in the same gesture space to formulate a single mathematical object or system. However, this 
taxonomy was created from gestures used by math teachers enrolled in courses at a selective university — which 
is arguably a very specialized population when it comes to gesture usage. In the present study, we extend this 
research to examine collaborative gestures among high school students in a diverse, urban school. We examine 
how these students use gestures across a variety of geometric tasks and engage in multi-modal analysis of two 
cases where gesture is used in collaborative ways. With this work, we seek to understand how gestures among 
learners can support collaborative problem solving. 


Theoretical Framework 
The gestures formed by collaborators engaged in sociotechnological contexts have long been integral to the 
study of computer-supported collaborative learning (Koschmann, 1996; Roschelle, 1992; Stahl, 2003). The 
theory of Gesture as Simulated Action (GSA; Hostetter & Alibali, 2008) posits that gestures arise during 
speaking when pre-motor activation, formed in response to motor or perceptual imagery, is activated beyond a 
speaker’s current gesture threshold. This threshold can vary depending on factors such as the current task 
demands (e.g., spatial imagery, task difficulty), individual differences (e.g., level of spatial skills), and 
situational considerations (e.g., social contexts). Nathan (2017) proposed an extension of GSA where learners’ 
actions and movements serve as inputs capable of driving the cognition-action system toward associated 
cognitive states through a bi-directional process. In other words, in addition to cognitive states giving rise to 
physical actions, directing learners to engage in certain physical motions may guide people toward certain 
cognitive states, prompting them to achieve intuitions and insights relevant to understanding and solving tasks. 
Although these theories forefront the individual’s cognitive processes, practice involves the 
coordination of many different inscriptions and representational technologies by differently-positioned actors 
whose actions occur across a range of social and physical spaces, which allows cognition to become distributed 
(Goodwin, 1995; Hutchins, 1995) and extended (Clark, 2008). Distributed, or extended, cognition observes that 
joint activity can be best understood with the system as the unit of analysis and accentuates how representations 
that exist inside and outside of the minds of individuals (like gesture) are used and transformed, as the system 
works to achieve a shared goal or plan. Students also often rely on gestures to establish and maintain 
intersubjectivity in order to operate conjointly in an effective manner (Nathan & Alibali, 2011). We define 
collaborative gestures as gestures that are physically and gesturally taken up by multiple learners. In this way, 


collaborative gestures hold a meaning that is explicitly dependent upon the gestures that have been enacted by 
interactional partners. These gestures represent an important case of distributed cognition in that they 
demonstrate how body actions can influence and extend over multiple learners’ mathematical reasoning in a 
way that is fundamentally different from traditional modes of exchanging mathematical information (i.e., speech 
and written work) (Walkington et al., in press; see also Flood, 2018). Proof is an important area for investigation 
because it is a complex and abstract area of math (whereas many studies of embodied math address primary 
grade topics). We examine collaborative gestures in the context of students’ proofs of geometric conjectures, a 
task that naturally integrates individual cognitive processes and collaborative processes. Students must ascertain 
for themselves the generalized properties of space and shape through investigations and logic (Harel & Sowder, 
1998). Further, proof is socially mediated in that the goal is to persuade others to accept one’s principled 
reasoning and justifications. It appears, then, that proof is “a richly embodied practice that involves inscribing 
and manipulating notations, interacting with those notations through speech and gesture, and using the body to 
enact the meanings of mathematical ideas” (Marghetis, Edwards, & Nufiez, 2014, p. 243). Our primary research 
questions are: How do high school students use gestures collaboratively when proving geometric conjectures in 
groups? How are collaborative gestures associated with mathematical problem solving? 


Method 
Participants included 51 high school students (20 female) from an urban Career and Technical Education high 
school in the Southern United States. At the school, 83% of students were economically disadvantaged, with 
57% Hispanic, 18% African American, 21% Asian, and 2% Caucasian. Of the 51 participants, 22 were in ninth 
grade and 29 were in tenth grade; 13 had not yet taken a high school geometry course. Participants were offered 
a $20 gift card and were pulled from class to participate in the study. 

Participants were placed in groups — there were 18 groups of 2 and 5 groups of 3. Groups played The 
Hidden Village — a motion capture video game — on a laptop hooked up to a Kinect™ camera. Following the 
introductory setup and calibration (Figure 1), players are introduced to the game narrative, and learn their 
player, lost and seeking to return home, will encounter tribal members whose actions that perform in order to 
earn amulets and advance their progress for a safe journey. The game then cycles through these encounters with 
a new character for each mathematical conjecture (8 in the current set up), following a Latin square design (see 
Table 1): Each new character performs poses that players must match closely enough (by trained Kinect 
standards) multiple times (3 times in this set up). Players then are presented with a new conjecture to which they 
respond True or False, and provide a (recorded) justification for their answers. They then select a justification 
among 4 multiple-choice options to advance to the next character/conjecture. 


Table 1: Eight game conjectures, with average performance metrics and gesture metrics (all in percentages) 


Any Collab. 


njectur Proof | Insight | Intuition 
COnECHIEE 8 i‘ Gesture | Gesture 


1. Given that you know the measure of all three angles 
of a triangle, there is only one unique triangle that can be 0 0 27 82 14 
formed with these three angle measurements 


2. If one angle of a triangle is larger than the second 
angle, then the side opposite first angle is longer than the 5 33 48 83 32 
side opposite the second angle. 


3. If you double the length and the width of a rectangle, 


then the area is exactly doubled. a 2 ae ua a 
4. Reflecting any point over the x-axis is the same as 

rotating the point 90 degrees clockwise about the origin. - - Pe Be oe 
5. The area of a parallelogram is the same as the area of 

a rectangle with the same base and width. 2 a ae a sai 
6. Diagonals of a rectangle always have the same length. 4 61 65 96 32 
7. Opposite angles of two lines that cross are always the 4 714 78 96 4] 
same. 

8. The sum of the length of two sides of a triangle is 5 33 29 73 14 


always greater than the length of the third side. 


TOTALS 9.6 42.4 51.4 85 26 
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00 lose to he noise and beg ose ht shinng rom Bhi the oes. You ‘each your 
| hand to pull away the brush when suddenly a hand reaches out toward yours! itis that of an old 
woman. She smiles gently and starts to pull you toward the light’s source. 


@ MEET the CHARACTERS 


R 


. > 
TWINS — WEAVER ELDER 


Al 


) 
{| 

= A - 
DANCER HUNTER HEALER WARRIOR 


Yau be wal oe age Sade oa ‘gensical chidren playing nearby call 0 
ve nw ue, aay “Helo sang! WI you lay 2 be game wah us? To 
decide 


(5 )RECEIVE SYMBOL @V=s Eee OO 


} a . a i 7 Explain why the statement is always true or is false 
Reflecting any point over the x-axis is the same as rotating the point 90 
degrees clockwise about the origin. Speak your answers out loud as we voice record your 
A) FALSE: Refecting a pont.overine 8) FALSE: Ireect te point (1-2) over responses: 
the x-axis, | get ‘This - 


the point (1, 2). 
coneevordek aecak 
‘ongia that s smaller than 90 degrees, 
rerefore tees Sse aoSera Bia eA Reflecting any point over the x-axis is the same as rotating the point 
90 degrees clockwise about the origin. 
©) TRUE: refet the point 1-1} over 0) TRUE: Reflecting a point over the 
the x-axis, | got the point (1, 1). This 
y to a 90-degree rotation 
} about the ongin, so these two actions 
are the samo. 


HINTS CYCLE 
WORKED EXAMPLE HINT SEQUENCE 


‘Two adjacent angles that form a line always sum to 180 degrees. 


‘The physical movernents that you performed in the game are Please perform the following movement with your ‘Suppose your arms form a horizontal fine. 
designed to help you understand math better. body as you read the statement out loud. 


Next, is an example that shows how certain physical 
movements are similar to understanding a specific math 
concept. 


‘Two adjacent angles that form a line always sum to 180 degrees. ‘Two adjacent angies that form a line always sum to 180 degrees. ‘Two adjacent angles that form a line always sum to 180 degrees, 


If your arms form an angle on one side, ‘and your arms form an adjacent angle on the other side, ears nes eee oer need 


CONJECTURE HINT SEQUENCE 


If you double the length and width of a rectangle, Expiain why the statement is always true or is false yeu cou elongth and wet of 2 ectangl on he aoa exacty 
then the area is exactly doubled. ‘Speak your answers out oud as we voice record your 
Vaal A) FALSE: Iie) beige Liar 


you double the length and width of a rectangie, then the area is 
‘exactly doubled, 


TRUE yoy ou eng and AL SE ze dua te rch rd 
wi Pe amas tro tna ng) 
res wah ote ae 


OUTRO 


d ogee eb deeb ae a 
from the emblem, illuminating the way forward, 


Figure 1. Flow chart of gameplay for The Hidden Village cycles through each of 8 mathematical conjectures. 


After completing the final conjecture, players in this version of the game receive hints for 4 of the 
previous conjectures (ordered in a 4x4 Latin square) explicitly relating prior conjectures to in-game movements, 
at which time players are re-prompted for True/False and justification responses The student controlling the 
game rotated through each group. 

Videotapes of groups of participants’ informal proofs while playing The Hidden Village provide us 
with rich, multimodal, presentations of students’ collaborative mathematical reasoning. The primary unit of 
analysis was the video clips, each of which was made of one group proving one conjecture. Clips were coded 
for (1) whether participants made any gestures while justifying the conjectures that represented or indicated 
mathematical objects or relationships, and (2) whether participants made any such collaborative gestures 
(defined earlier) while justifying the conjectures. Clips were also coded according to whether students were able 
to: correctly determine the truth value of each conjecture during their initial snap judgment (our measure of 
intuition); show understanding of key mathematical ideas and relationships underlying each conjecture (insight 
measure); and construct a valid, logical deductive mathematical argument to justify or disprove each conjecture 
(proof; see Harel & Sowder, 1998; see example in Table 2). Collaborative gestures were coded by type (mirror, 
echo, addressee, joint, and alternate; Table 3). Finally, all collaborative gesture clips were re-viewed to select 
clips for further analysis that pushed the boundaries of and offered worthwhile extensions for the collaborative 
gesture taxonomy previously developed using data of math teachers proving conjectures. Multimodal analysis 
(see McNeill, 1992) was used to examine these clips. We present two of these clips in the next section. 


Table 2: Example coding criteria for conjecture performance measures 


Diagonals of }TRUE | TRUE because opposite sides are TRUE because diagonals form two right 


congruent and/or are parallel and/or | triangles that are congruent (by SAS), since 
the angles are right angles and/or the | opposite sides of a rectangle are congruent, so 
rectangle is symmetric. their hypotenuses must be congruent. 


Table 3: Collaborative gesture coding criteria 


Category Description 


Mirror One student makes a gesture, and then a second student makes the same (or similar) gesture at 
the same time. Only the second gesturer is coded as collaborative -mirroring. Must be 
evidence that the person doing the mirroring was looking at the original gesturer. 


Mirror and One student mirrors the gestures of another student, but then before dropping their hands 
Build builds upon the mirrored gesture with a new gesture that the original student is not making. 
Mirror- One person is gesturing, and then a second person anticipates a gesture they are about to do 
Anticipation (correctly or incorrectly) and makes that gesture. 

Echo One student makes a gesture, and then a second student makes the same gesture (or a very 


similar gesture) afterwards. Must be evidence that the person doing the echoing was looking at 
the original gesturer when the original gesturer originally made the gesture. 


Echo and One student makes a gesture, and then a second student makes part or all of the same gesture 

Build (or a very similar gesture) afterwards. However, the second student also changes or adds to the 
gesture in some way, in addition to echoing the original gesture. 

Addressee One person is speaking (may or may not be gesturing), and another person gestures to show 


what that person is talking about while they are still talking. 


Joint Gesture Multiple people manipulate, point to, or formulate a single mathematical object or a unified 
set of interacting mathematical objects. Each gesturer who is included in the joint gesture is 
coded as doing a collaborative gesture. 


Alternate & One learner gestures their understanding, and then another learner follows up, building on 
Build their reasoning using a different gesture, but generally agreeing with their viewpoint 
Alternate & One learner gestures their understanding, while another learner shows how their reasoning is 
Redirect different using a different gesture, potentially refuting the original person's reasoning 
Results 


Participant groups gave the correct intuition 51.4% of the time, had the correct insight 42.4% of the time, and 


gave a valid oral proof 9.6% of the time (Table 1). They made any gestures on 85% of attempts to prove or 
disprove conjectures and made collaborative gestures specifically on 26% of proof attempts (Table 1). Of the 
45 instances of collaborative gestures, 25 were echoing gestures, 11 were mirroring gestures, 9 were alternating 
gestures, and 7 were joint gestures. Table 1 shows how these metrics varied by conjecture. 

Figure 2 shows the relationship between gesture and outcomes. Performance measures varied by how 
student groups gestured. Student groups had the correct intuition 27% of the time when they did not gesture, 
compared to 56% of the time when they made any gesture, and 70% of the time when they made collaborative 
gestures. Similarly, student groups had the correct insight 12% of the time when they did not gesture, compared 
to 48% of the time when they made any gesture, and 72% of the time when they made collaborative gesture. For 
the proof outcome, however, accuracy was low enough overall that there was little differentiation (8% no 
gesture, versus 10% any gesture and 9% collaborative gesture). These relationships do not imply causation 
between collaborative gesture and successful reasoning, but show how gestures, especially collaborative 
gestures, have a powerful association with students’ mathematical intuition and insight. Student groups who 
made no gestures were least likely to give a correct intuition or insight, while groups who made collaborative 
gestures specifically were most successful. 
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Figure 2. Association between outcomes (intuition, insight, proof) and gesture categories. Error bars represent 
standard error of the mean. 


Our first transcript (Figure 3) shows two male students proving the conjecture that diagonals of a 
rectangle have the same length. The student on the left (P1) is controlling the game and gives an oral proof for 
the conjecture, which includes gesture. One interesting thing about this exchange is that P2 is literally silent the 
entire time, but continuously communicates significant mathematical information to P1 only using gesture. P2’s 
gestures initially resemble addressee gestures (Alibali & Nathan, 2012), but then the two of them are gesturing 
simultaneously, with P1 both echoing and mirroring P2. P1 and P2 then reach intersubjectivity around the idea 
of diagonals being the same length using gesture as their primary mode of collaborative exchange of ideas. As 
P1 begins the proof, P2 begins a rhythmic, repeated series of gestures in the air tracing the sides of a rectangle 
(Lines 1-3). On Line 2, Pl echoes this gesture series, but then adds diagonal gestures instead of strictly 
horizontal and vertical gestures (Lines 3-4), noting that one diagonal is just a reflection in a different direction 
of the other. In response to this insight, P2 pauses his gesture sequence, nods at Pl, and then makes his own 
diagonal gesture, echoing P1. P1 attends to this gesture with his gaze as P2 nods again, completing their proof. 
This transcript demonstrates how complex mathematical reasoning is communicated via gesture, and the power 
of collaborative gestures for supporting intersubjectivity, even in the absence of collaborative speech. 

Our second transcript (Figure 4) shows two males students (P4 and P5) and one female student (P3) 
reasoning about the conjecture stating that the longest side of a triangle is across from the largest angle. This 
transcript shows an interesting and extended use of joint gestures to create embodied mathematical objects, and 
one larger observation from the dataset was that such gestures can open up access to mathematical ideas. Rather 
than there being someone controlling the computer, writing on the paper, or handling the manipulatives, anyone 
can make gestures at any time, and they are highly visible to all group members. However, this case shows that 
collaborative gestures can still exclude. Before the transcript began, PS made a triangle gesture with his fingers, 
and tried unsuccessfully to get P4 to attend to his gesture. At the beginning of the transcript, P4 forms a triangle 
with his fingers (Line 6), and then P5 touches two of the sides of this triangle with his fingers in turn (Lines 7- 


9), referencing the angles as they reason through the relationships together. P5 keeps his finger on P4’s triangle 
gesture for a full 11 seconds (Line 10), and although P4 never looks at the joint gesture they are formulating, he 


1 Pi: “Um, okay, so the length is going to 
stay the same because... 

((P2 traces horizontal and vertical lines in air 

with two hands, tracing rectangular shapes. 

P25 gaze is on P].)) 


2 ~P1: like if you think of it- if you think of 

it um...” 

((PI extends two arms above him with pointer 
Singers extended and touching. He moves both 
Singers apart, brings them down, and connects 
them back together back and forth repeatedly. 
P1'’s gaze is following hand movements, but 

briefly flashes to P2. At the same time, P2 
continues his pattern of horizontal and 
vertical lines in air, with gaze on partner:)) 


3 Pl: “The length, the length isn’t changing. ..~ 
((P1 moves hand back and forth from top left to 
bottom right. Pl'’s gaze is on partner: P2 
continues tracing lines in air, and then pauses his 
gesture and nods at P], P2’ gaze is on partner:)) 


4 P1:~.... it's more the direction is 
changing. 

((PI moves hand back and forth from top right 

to bottom left. Gaze stays on hands. P2 pauses 

and appears to be listening, gaze on P1.)) 


5 Pl: “You know? So yeah. They will have 
the same length you know so yeah.” 

((P2 places all four index fingers and thumbs 

together by his stomach, and then draws 

right arm upward and diagonally. P1'’s gaze 

is on P2. P2 nods to P1.)) 


Conventions: Double parentheses describe gaze and gestures. Bolded text shows speech that co-occurred with 


gestures. Arrows show movement (red) or direction of gaze (blue). Numbers show the order in which gestures 
occurred. Method adaptedfrom Goodwin (20053). 


Figure 3. Multi-modal transcript of two students proving that diagonals of a rectangle are congruent 


gets tactile feedback in that he can feel where on his gestured triangle PS is pointing. The gesture remains 
anchored there as a sort of diagrammatic representation that they can continue to refer to for further scrutiny. 
P3, briefly glances at the joint gesture (Line 7) but remains excluded from P4’s & P5’s joint embodied 


reasoning. After the transcript ends, P4 and P5 continue reasoning together using collaborative gestures, 
excluding P3, who eventually draws on her hand to try to make sense of the relationships by herself. 


6 P4: “The first triangle... 

((P4 holds both index fingers and thumbs out and 
connects index to index finger and thumb to thumb. 
Gaze is on screen.)) 


7 P5: This one... 

((P5 uses pinky finger to touch P4’ index finger on 
the right. P4 and P5 gaze at screen, P3 briefly gazes 
at their hands.)) 


8  P4: “is larger than the second angle...” 
9 P5: “This one...” 


((P5 uses pinky finger to touch P4’ index finger on 
the left. Each gaze is on the screen.)) 


10 P4: “then the side opposite of the first triangle 
“_.. 1s longer than the side opposite- wait, what?” 
((P4 and PS hold gesture, and then both drop it.)) 


11 P3: “I don't get it.” 


Conventions: Double parentheses describe gaze and gestures. 
Bolded text shows speech that co-occurred with gestures. Arrows 
show movement (red) or direction of gaze (blue). Method 
adaptedfrom Goodwin (2005). 


Figure 4. Multi-modal transcript of three students exploring the conjecture that the largest angle of a triangle is 
across from the longest side. 


Educational and Scientific Significance 

This study extends research on gesture and embodied mathematical cognition by empirically describing 
students’ collaborative gestures in a sociotechnological context. First, we documented the ways in which high 
school students use gesture collaboratively and show how gestures and collaborative gestures are each 
associated with different problem-solving outcomes. Second, we show important ways in which students can 
use joint, embodied action to explore the truth and generalizability of geometric concepts, establish common 
ground through shared body-based resources, and build — literally — on each other’s reasoning and enacted 
representations. Students use gestures collaboratively in powerful ways that enhances the quality of their 
reasoning. Having students sense-make together about mathematics using their bodies has the potential to open 
up access to mathematical reasoning and argumentation. In the present study, students enlisted their bodies and 
drew on the bodily actions of their peers in collaborative ways within an embodied gaming environment where 
typical media — pencil/paper, dynamic geometry software, measurement tools, and manipulatives — were not 
available. An important question stemming form this work is how such embodied collaboration can be 
facilitated. Geometric proofs may be a particularly rich domain for fostering collaborative gestures since these 
activities invoke spatial reasoning and motor systems, as well as socially mediated exploration and persuasion. 
We believe next steps include exploring task designs and other sociotechnological ways to engage students in 
joint, embodied reasoning, which activate collaborative gesture as another resource for their extended shared 
cognition. 
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